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Abstract 

In this paper, we consider a simple linear exponential quadratic Gaussian (LEQG) 
tracking problem for a multi-agent system. We study the dynamical behaviors of the 
group as we vary the risk-sensitivity parameter, comparing in particular the risk averse 
case to the LQG case. Then we consider the evolution of the performance per agent as 
the number of agents in the system increases. We provide some analytical as well as 
simulation results. In general, more agents are beneficial only if noisy agent dynamics 
and/or imperfect measurements are considered. The critical value of the risk sensitivity 
parameter above which the cost becomes infinite increases with the number of agents. 
In other words, for a fixed positive value of this parameter, there is a minimum number 
of agents above which the cost remains finite. 

1 INTRODUCTION 

With the rapidly growing interest in sensor networks and distributed control systems, a large 
body of litterature in recent years has focused on new and old engineering problems that 
these networks pose. Examples include consensus problems in various forms (see [1] and 
the references therein), control design with pre-specified information passing structure and 
control under communication constraints (e.g. [2, 3]), designing communication schemes in 
ad-hoc networks [4], etc. 

In some cases, such as in signal processing and control for distributed parameter systems 
(e.g. [5, 6]), the distributed architecture of the system is imposed by the task in mind. But 
in a large number of situations, especially involving mobile robotic networks, the designer 
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has a significant freedom in clioosing tlie degree of distributivity of the system, the number 
and type of agents to use, etc. Yet relatively little work has focused on understanding 
quantitatively the benefits of multi-agent systems over centralized systems for various tasks, 
or the impact of the number of agents on the performance of the system, when this choice 
is available. 

For a given task at hand, it is important to study these problems, because they influence 
most of the system design. For example, scalability is a relative notion. If we find that no 
significant improvement, or worse a decrease in performance is to be expected if we employ 
thousands of agents instead of tens or hundreds, it is probably better to design algorithms 
with good performance that are not necessarily the most decentralized. Moreover, it is clear 
than more agents is not always beneficial. For example, the capacity of a wireless sensor 
network is expected to decrease as the number of nodes increases [7]. A similar issue arises 
in mobile robotic networks where conflict free vehicle routing is necessary and congestion 
increases with the size of the network [8]. 

In some recent work researchers have obtained asymptotic performance scalings for certain 
multi-agent systems. Let us mention [9, 10] on various dynamic routing problems. One 
can also find in the computer science litterature a large number of papers focusing on the 
minimum number of agents necessary to perform certain tasks: for instance, the minimum 
number of pursuers to catch an evader [11], or the minimum number of guards for an art 
gallery [12]. 

In this work, we study a simple multi-agent tracking problem using linear quadratic control 
tools, which are well suited for performance evaluation. We focus on the evolution of the 
performance per agent, as the number of agents increases. Numerous papers describe multi- 
agent architectures for tracking and estimation, the simultaneous localization and mapping 
problem, etc., focusing on the various subroutine designs described earlier (e.g. [13]). Be- 
cause of the link between risk-sensitive control and differential games, our application is also 
related to some recent work on multi-agent pursuit-evasion games [14]. The model we con- 
sider is simpler, but our focus is on obtaining insight regarding the performance asymptotics 
of the system, which these references do not usually discuss. 

The rest of the paper is organized as follows. Section 2 describes our model and section 3 the 
general form of its solution. We find in particular that a standard LQG formulation might 
not provide a satisfying cooperative control law for the group of agents, as the individual 
controllers decouple, essentially because of the certainty equivalence principle. We can obtain 
coupled control laws however if we consider risk-sensitive agents. Intuitively, we expect 
the set of risk-sensitive parameters where tracking is possible, or the robustness to model 
disturbances of our system, to increase with the number of agents. This is related to the 
earlier observation that a minimum number of agents can be necessary in certain pursuit- 
evasion games. Section 4 provides some analysis of this aspect. Increasing the number of 
agents becomes critical in our scenario only in the case of noisy dynamics and, of course, 
noisy measurements since more agents obtain a better state estimator. We give elements of 
analytical performance analysis as well as simulation results in that section as well. 



2 Basic Model 



We will consider the following tracking problem. There is an evader moving randomly in W^, 
subject to the linear dynamics 

Xe = AXe + GWe, (1) 

where We is a standard c?-dimensional white Gaussian noise, and we let W = GG' . 

We have n identical pursuers, which are also described by linear systems, with the same A 
matrix for simplicity: 



X. 



AxpA + BupA + VeFwpA, i = 1,. . . ,n, 



p,i ^'-^p,i I "'p,i I V ^p,i} 

where standard dimensional white Gaussian noises, independent between different 

agents and from We- e is a parameter that will tend to in parts of the analysis later on. Let 
Z = FF'. The initial positions of the agents are also independent d-dimensional Gaussian 
random vectors. 

Each pursuer incurs a running cost 

which is quadratic in the difference between its state and the state of the evader. There is 
also a running control cost \u'p^Rup^i for each agent, where i? is a positive definite matrix. 
Alternatively, the n mobile agents are tracking the center of an ellispoid moving randomly 
in W'- according to (1), whose shape is known and given by a positive semi-definite matrix 
Q. A motivation for this model is cooperative soaring and tracking ascending currents for a 
team of Unmanned Aerial Vehicles. 

Each agent has access to a relative measurement: 

Vp,! G{xp^i Xg^ -\- Hvp i, i 1, . . . , 77., 

with fp j a standard d-dimensional white noise. We let V = HH' and assume V to be positive 
definite. The measurement noise processes of the different agents are independent, and also 
independant of the various noises in the dynamics. For example, the agents could obtain 
noisy measurements of the gradient of the quadratic cost function (2), in which case C = Q. 
We will also consider the perfect measurement case, where y = x, subject to no measurement 
noise. 

We define Xi = Xp^i - Xe, and the aggregate vectors x = [x[, x'J, y = [y^ ^, . . . , y'^J, 
u = [Wp 1, . . . , Mp^^]', Wp = [w7p 1, . . . , Wp^„]', V = [v'p^i-, . . . , fp^„]'. We will also use the Kronecker 
product of matrices, which we recall, li A = [aij] and B are matrices, then we have by 
definition: 

CLiiB . . . CLlnB 



A®B 



O-mlB . . . CtjYmB 



and we will use the property: 



{A ®B){C®D) = AC® BD. 

The cigcnvahics oi A® B are {Aj/Xj} and the corresponding eigenvectors {xi ® yj}, where 
{Ai}, {xi}, {/ij}, {Vj} are the eigenvalues and eigenvectors of A and B respectively. Hence if 
A and B are positive definite matrices, so is A ® B. Denote by /„ the n xn identity matrix, 
by In the column vector of ones of size n, and by £"„ = Inl'n the n x n matrix of ones. Let 

A„ = /„ ® A, Bn = In® B, Fn = In® F, ^ I„ ® Z, 

Cn^In® C, Hn^In® H, V„ ^ ® V, 

be the block diagonal matrices describing the dynamics of the group of n pursuers. Finally, 
let Gn — In G and the corresponding spectral density matrix be 



Wn = 



W 



GnG'n 



W 



W 



w 



w 



Then the evolution of the relative states is described by: 

X = AnX + BnU + \^FnWp - GnUUg, x{0) = Xq, 

with observations 

y ^CnX + HnV. 

We will assume that xq is a Gaussian random vector with mean xq and covariance matrix 
Eo y 0. 

2.1 Basic Example: Simple Integrators 

In an example used repetedly in the following, the pursuers are simply n integrators and the 
evader follows a standard brownian motion: 



We. 



i.e., we have the dynamics 



X = u + \/eWr, 



We 



We 



We will also take C = H = I^, i.e.. 



y ^ c + v. 



3 Risk-Sensitive Tracking 



Let Qn = In^Q and Rn = In® R- We formulate the tracking problem as an infinite horizon 
linear exponential quadratic Gaussian (LEQG) problem: 



The ^ factor is due to the fact that we want to obtain a measure of performance per agent. 
If > we consider risk-averse agents, if ^ < the agents are risk-seeking and in the limit of 
6' = 0, the agents are risk-neutral and we recover the standard LQG formulation. There is a 
very large litterature on the LEQG problem, as well as on more general risk-sensitive control 
problems, that we cannot review here. Suffices to say that the LEQG problem was introduced 
in the engineering litterature and solved in the full information case by Jacobson (who did 
not use the logarithmic transformation) [15]. Links with differential games are emphasized 
already in that paper, and the relationship to control is considered in [16, 17, 18]. The 
risk-sensitive state estimator is not the Kalman filter, and robustness properties of this filter 
have been explored for example in [19]. An important motivation for the early work of 
Jacobson in the full information case was to obtain a controller which is not independent of 
the noise in the dynamics, as arises in the LQG case because of the certainty equivalence 
principle. This motivation translates in our case into obtaining controllers for the pursuers 
which are not independent of the known characteristics of the random motion of the evader, 
certainly a desirable feature. 

The LEQG output feedback solution over an infinite horizon is described for example in 
[18, 20]. We start by considering the full state feedback problem or perfect measurement 
case, that is, y = x. Introduce the quantity 



mmimize 



J 




and consider the generahzed algebraic Riccati equation (GARE) 

A^X + XAr,-X SM X + = 0. 



(3) 



Now define the quantity 



e*{n) = sup {0 G M : the GARE (3) admits a 
positive definite solution X„ . 



(4) 



Assume that {A,B) is controllable and {A,Q) is observable. Then we have that 6*{n) is 
positive, and for all 6 < 6*{n), the LEQG problem with perfect state measurements admits 
an optimal state-feedback solution 



u = ~inR~'B'^Xn,e)x, (5) 

with the optimal cost being 

J*{e,n) = TT{{Wn + eZ^)X^,e). (6) 

Furthermore the feedback matrix An — nBnR~^ B'^Xn^ is Hurwitz. In the following we will 
omit to write the dependence with respect to 9 of the solution of the GARE, and write just 



Now consider the output feedback solution for the problem with imperfect measurements 
described 
quantity 



described earher. Recall that we assumed V > and let V„ = J„ ® V. Introduce the 



U9) = c'nV-'Cn-e^, 

n 

and consider the two GAREs (3) and 

y< + AnY - YTM y + {Wn + tZn) = 0. (7) 

Define the quantity 

9*j{n) = sup{6 G M : the GAREs (3) and (7) admit minimal 
positive definite solutions X„ and Yn, respectively, and 
the matrix I — OYnXn has only positive eigenvalues}. 

For 6 < 9*j{n)^ introduce the filter 

dx O 

— = (A„ + OYn^) x + BnU + Yr,Cy~\y - C„ x), 
at n 

Xq = Xq. 

Let X = {I — 9YnXn)~^x. Then one can compute that that x is generated by the following 
differential equation: 

fj Of 

— = {An- s^{e)x^) x + {i- eYnXn)-' B u 

at 

+ (/ - eYnXn)-^YnCy-\y - C„x), 

where u = u + nR~^B'nXnX. 

Now suppose that the pairs {A,B) and [v^-F„, —(?„]) are controllable. For the second 
condition, in practice we will assume (A, F) to be controllable. Also, assume that the pairs 



(a) Risk Seeking Agents. (b) Risk Neutral Agents. (c) Risk Averse Agents. 

Figure 1: Trajectories for Risk Sensitive Agents. 



[A, C) and [A, Q) are observable, and Sq < Yn, then for all 6 < 0*j{n), the optimal controller 
is given by 

u* = -riR-'B'^X^il - OY^X^yH = -nR-'B'^X^x. 

The optimal cost is 

j;(0,n) = Tr + Y^C'^V-'CnY^X^ (/ - OY^X^y' ^ . (8) 

3.1 Application to the basic example 

It is interesting to visualize the trajectories of the agents in a simple case of the basic example. 
We consider a situation with perfect information, and e = 0. The agents are tracking an 
evader according to the model of random motion described earlier, but in this simulation 
the evader in fact remains immobile at the origin. The trajectories are shown on Fig. 1, for 
risk-averse, risk-neutral and risk-seeking agents, and show different qualitative convergence 
behaviors. 

The risk-neutral trajectories are essentially trivial in the sense that the controllers are totally 
decoupled and the control law for one agent depends only on its separation from the target. 
In the risk-seeking and risk-averse case however, the controllers become coupled, and each 
agent needs information about the position of all the other pursuers as weel as the target. 
In the risk averse case, two set of agents starting at different distances from the target try 
to track it from both sides. Intuitively, the overshoot is due to a pessimistic behavior which 
leads them to give more importance to the case where the target moves away from them. 



4 Performance Analysis 

Equations (6) and (8) give the cost per agent when the group of pursuers consist of n agents. 
In this section we study how this individual cost evolves as the number of agents increases. 



4.1 Perfect State Measurements 



4.1.1 LQG solution 

The LQG solution is obtained for = (risk neutral). In this case we can rewrite the control 
GARE (3) as: 

A'^X + XAn-XnBnR-^B'^X+-Qn = Q. (9) 

n 

Due to the block diagonal form of the matrices, the equation decouples by agents. With 
our controllability and observability assumptions, (9) has a unique positive definite solution 
([21], corollary 13.8). In particular, if Xi is the positive definite solution for one agent, i.e., 

AXi + XiA - XBR-^B' X + Q = 

then by unicity the solution for n agents is verified to be 

n 

The controller for the group, u = —{In®R^^B'Xi) x is decoupled as well, that is, each agent 
considers only the state difference between himself and the target, but not the states of the 
other agents. 

The cost per agent in the perfect information case is then 

J*(0, n) = Tr((l^„ + e Z„) X„) = -Tr(K ® WX^ + /„ ® e ZX^) 

n 

= Tr((iy + eZ)Xi) = J*(0,1), 
that is, the cost per agent is independent of the number of agents. 



4.1.2 Risk-Sensitive Solution with no dynamics noise 

When 9^0, some coupling between the controllers of the agents is introduced through 
the matrix Wn- This is a desirable feature as intuitively we would like the agents to take 
advantage of the fact that they can cooperate. We will take e = here, which still leads to 
a nonsingular problem in the perfect information case. 

Consider first the case of one agent. We can always write the solution as a pertubation of 
the LQG solution, i.e., as 

Xi = Xi + Xi , 

where Xi is the solution to (3) for ri = 1 and Xi is the solution to (9) for n = 1. 



Proposition 4.1. For n agents and e = 0, the LEQG problem with perfect state measure- 
ments admits an optimal state-feedback solution solution (5) with the solution to (3) given 
by 

X„= -/„®Xi + — (10) 
n 



Proof. By definition we liave tliat Xi and Xi verify: 

AXi + XiA - XiBR-^BXi + Q = (11) 
A'(Xi + Xi) + (Xi+Xi)A 

- (Xi + Xi){BR-^B' - dW){Xi + Xi) + g = 0. (12) 

We try tliis candidate solution in tlie GARE (3) for e = 0. Then by a straightforward 
calculation, using the identity E"^ = nEn, we obtain: 

^ J„ ® (A'Xi + XiA - XiBR-^BXi + Q) + 
n 



A'(Xi+Xi) + (Xi + Xi)A 
- (Xi + Xi){BR-^B' - eW){Xi + Xi) 
-{AXi + XiA - XiBR-^B) 
= 0. 

Indeed, in the first term we have (11) and in the second term we have the difference between 
(12) and (11). 

Moreover, this solution X^ is positive definite and stabilizing. This follows from the fact 
that the eigenvalues of 

/„ ® M + — ® X, 
n 

for any diagonalizable matrices M and (M + X), are the eigenvalues of M with multiplicities 
{n — 1) as well as those of (M + X). A set of corresponding eigenvectors are yi ® Wj, where 
the Hi are (n — 1) vectors spanning the kernel of En (which is symmetric) and the Wj are 
eigenvectors of M, and 1„ ® Vi, for Vi eigenvector of (M + X). It is then easy to see that 
the eigenvalues of X„ are thoses of Xi and Xi, and those of An — nBnRn^B'^Xn are the 
eigenvalues of A — BR~^B'Xi and A — BR~^B'Xi. But Xi and Xi are stabilizing and 
positive definite. □ 



Remark. In particular, we see that the critical value 6* (n) is independent of n in the perfect 
information case when e = 0. 



We can then compute the cost of this solution to the risk-sensitive tracking problem, for 
e = 0. We get 

r{e,n) = TiiWnXn) = Tr(-E„ ® wx^ + \eI ® WX^) 

n 

= Tr(H^(Xi + Xi)) = J*(^,l). 

Hence, as in the LQG case, the cost per agent in the case e = is independent of the number 
of agents. 

4.1.3 Risk-Sensitive Solution with noisy dynamics 

The risk-sensitive solution in the perfect measurement case seems to be most relevant when 
noise is present in the dynamics of the agents. In this case, the cost-per-agent is not inde- 
pendent of the number of agent any more, and moreover, the critical value of the parameter 
9*{n) increases with n. Another way to say this is that for a fixed value of risk aversion 6'*, 
there is a minimum number of agents that are necessary to obtain a finite cost. 

We only provide a numerical illustration of this fact. Fig. 2 shows the cost per agent in the 
basic example with e = 0.1. We set 9* = 0.97. In this case, the cost per agent is finite only 
if 4 agents or more are used. The lower bound is the cost per agent for e = 0. 

Cost per agent (6* = 0.97) 
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Figure 2: Cost per agent in the basic example with noisy dynamics, under perfect measure- 
ments. The cost is infinite for less than 4 agents. 




4.2 Imperfect State Measurements 



In the case of imperfect measurements, there is one obvious advantage of using more agents. 
As the number of agents taking measurements with independent noise sources increases, a 
better state estimate can be constructed. In this section, we consider the imperfect mea- 
surement tracking problem, with the simplifying assumption A = 0. Note that in this case, 
the matrix {An, ^/e-Fn — Gn) is not stabilizable when e = 0, resulting in an only marginally 
stable filter. Hence we have to keep a small noise term in the calculation, but in the analysis 
we will consider the dominant part as e — 0, for which a clear answer is available. 



4.2.1 LQG solution 

For ^ = (risk neutral), the filter algebraic Riccati equation (7) for A = becomes: 

-YC'nV-'CnY + {Wn + eZn) = 0. (13) 

Let Yi be the positive definite solution (unique in the LQG case) of the Riccati equation 
(13) for one agent, i.e., 

-Fi C'V-^C Yi + {W + eZ) = 0. 

Then we have 

(^ ® ri)(/„ ® CV-'C)i^ (g) Yi) = (^) ®{W + eZ) 
= En0W + eEn0 Z, 

and so as e ^ 0, the solution to the n agent problem, for which the right hand side in the 
previous equation is En<gW + ein ® Z, approaches F„ = ^ ® Y^. 

The total cost is 

j;(0,n) = Tr(F„^ + {W^ + eZ„)X„). 

n 

We have already seen that if e = 0, the second term of this expression becomes independent 
of n. Hence, as e — 0, the cost approaches 

j;(0,n) ^ iTr|(^®ri)(/„®g)l +Tr(iyXi) 
n \ n 



^^Tr{riQ} + Tr(W^Xi) 



In conclusion, as the number of agents increases, the tracking performance per agent con- 
verges to the control performance for one agent at rate Xj^fn, due to a better estimation 
performance only. This is intuitively expected from our understanding of the asymptotic 
normality of maximum likelihood estimators. However, if we consider the more general dif- 
fusion process with A ^ and e 7^ 0, it is not clear if this asymptotic rate of convergence 
still holds. 



4.2.2 Risk-Sensitive Solution 



Again let us consider the case A = and e 0. In the limit e = 0, the solution to the 
control GARE is given by (10) and this equation does not influence the critical value of the 
parameter 6}. The fllter GARE is: 



r(J„ ® (CV-^C - -Q))Y = En®W + tIn®Z. 
n 



(14) 



Hence as in the LQG problem, we see that as e — > 0, the solution to this equation approaches 

Er, 



n 



(15) 



but an essential difference is that now, Yi „ is the solution to the equation: 

Y{CV-^C - -Q)Y = W + tZ, 
n 



(16) 



which is not the single agent equation, but depends on n through the parameter 9/n. Fig. 3 
provides some experimental results on the performance of the multi-agent system in various 
cases considered above. 
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Figure 3: Cost per agent in the basic example with noisy dynamics and observations 
(P.M.=perfect measurements, I.M.=imperfect measurements. A=0). In the case e = 0.1 
and noisy observations, the cost is inflnite for less than 10 agents. 



If the constraint that / — ^1^X„ must have only positive eigenvalues were not present, from 
(16) 0*j{n) would increase linearly with n. Now it is easy to check from (10) and (15) that 
as e 0, YnXn approaches 

E 

En/n has eigenvalues 1 and 0, so the condition as e — becomes 

where p denotes the spectral radius. We have indicated the dependence of Yi,„ and Xi on 9 
in this condition. Fig. 4 shows the evolution of the critical value 0*j{n) with the number of 
agents, computed for the basic example. We obtained similar experimental results for e ^ 0. 
However if A 7^ it seems that we can obtain different behaviors. 

Note that in the various cases considered above, when an analytical solution could be ob- 
tained it always implied that for solving the n agent problem, we only need to solve a Riccati 
equation of the same size as for one agent. Hence significant computational reduction can 
be achieved by taking advantage of the symmetries present when considering homogeneous 
agents. 
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Figure 4: Evolution of 0*j{n) with the number of agents in the basic example (e ~ 0, A=0). 



5 CONCLUSION 



In this work we have considered a basic tracking task to be performed cooperatively by a 
team of mobile sensors. We have given some elements of analysis concerning the influence of 



the size of the group on the individual performance. Intuitively, there seems to be much gain 

to be obtained by multi-agent systems in terms of robustness and in a risk-sensitive context, 
and we believe that more work in this direction is needed. In any case, it is clear that there 
is a need for a better understanding of the beneficial role of distributed architectures, when 
this aspect is under the control of the system designer. 
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